Papers with TyDi QA
Canine: Pre-training an Efficient Tokenization-Free Encoder for Language Representation (2022.tacl-1)
Copied to clipboard
| Challenge: | End-to-end neural models have replaced the traditional pipeline and require an explicit tokenization step. |
| Approach: | They propose a neural encoder that operates directly on character sequences without explicit tokenization or vocabulary and a pre-training strategy that optionally uses subwords as a soft inductive bias. |
| Outcome: | The proposed model outperforms a comparable mBert model on a multilingual benchmark by 5.7 F1 on the TyDi QA benchmark. |
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020.tacl-1)
Copied to clipboard
Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, Jennimaria Palomaki
| Challenge: | Existing models for multilingual modeling are based on a set of typological features that are used to express meaning in languages such as English. |
| Approach: | They present a question-answer-typed question-referenced dataset that covers 11 typologically diverse languages with 204K question-and-answered pairs. |
| Outcome: | The proposed dataset covers 11 typologically diverse languages with 204K question-answer pairs. |
Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to zero-shot transfer knowledge from rich-resource to low-resourced languages are limited due to linguistic discrepancies in different languages. |
| Approach: | They propose a multilingual MRC framework equipped with a Siamese Semantic Disentanglement Model to disassociate semantics from syntax in models learned by multilingual pre-trained models. |
| Outcome: | The proposed model disassociates semantics from syntax in multilingual models. |
Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | Existing pretrained language models have solved reading comprehension benchmarks, but datasets with information-seeking queries remain challenging. |
| Approach: | They analyze why answering information-seeking queries is more challenging . they manually annotate 800 unanswerable examples across six languages . |
| Outcome: | The proposed model outperforms human annotators on 800 unanswerable examples across six languages. |